Autonomic Nervous System Dynamics for Mood and Emotional-State Recognition: Wearable Systems, Modeling, and Advanced Biosignal Processing

This thesis aims at investigating how electrophysiological signals related to the autonomic nervous system (ANS) dynamics could be source of reliable and effective markers for mood state recognition and assessment of emotional responses. In-depth methodological and applicative studies of biosignals such as electrocardiogram, electrodermal response, and respiration activity along with information coming from the eyes (gaze points and pupil size variation) were performed. Supported by the current literature, I found that nonlinear signal processing techniques play a crucial role in understanding the underlying ANS physiology and provide important quantifiers of cardiovascular control dynamics with prognostic value in both healthy subjects and patients. Two main applicative scenarios were identified: the former includes a group of healthy subjects who was presented with sets of images gathered from the International Affective Picture System hav- ing five levels of arousal and five levels of valence, including both a neutral reference level. The latter was constituted by bipolar patients who were followed for a period of 90 days during which psychophysical evaluations were performed. In both datasets, standard signal processing techniques as well as nonlinear measures have been taken into account to automatically and accurately recognize the elicited levels of arousal and valence and mood states, respectively. A novel probabilistic approach based on the point-process theory was also successfully applied in order to model and characterize the instantaneous ANS nonlinear dynamics in both healthy subjects and bipolar patients. According to the reported evidences on ANS complex behavior, experimental results demonstrate that an accurate characterization of the elicited affective levels and mood states is viable only when non- linear information are retained. Moreover, I demonstrate that the instantaneous ANS assessment is effective in both healthy subjects and patients. Besides mathematics and signal processing, this thesis also contributes to pragmatic issues such as emotional and mood state mod- eling, elicitation, and noninvasive ANS monitoring. Throughout the dissertation, a critical review on the current state-of-the-art is reported leading to the description of dedicated experimental protocols, reliable mood models, and novel wearable systems able to perform ANS monitoring in a naturalistic environment

Inferring directionality of coupled dynamical systems using Gaussian process priors: Application on neurovascular systems

Dynamical system theory has recently shown promise for uncovering causality and directionality in complex systems, particularly using the method of convergent cross mapping (CCM). In spite of its success in the literature, the presence of process noise raises concern about CCM’s ability to uncover coupling direction. Furthermore, CCM’s capacity to detect indirect causal links may be challenged in simulated unidrectionally coupled Rossler-Lorenz systems. To overcome these limitations, we propose a method that places a Gaussian process prior on a cross mapping function (named GP-CCM) to impose constraints on local state space neighborhood comparisons. Bayesian posterior likelihood and evidence ratio tests, as well as surrogate data analyses are performed to obtain a robust statistic for dynamical coupling directionality. We demonstrate GP-CCM’s performance with respect to CCM in synthetic data simulation as well as in empirical electroencephelography (EEG) and functional near infrared spectroscopy (fNIRS) activity data. Our findings show that GP-CCM provides a statistic that consistently reports indirect causal structures in non-separable unidirectional system interactions; GP-CCM also provides coupling direction estimates in noisy physiological signals, showing that EEG likely causes, i.e., drives, fNIRS dynamics

Robust Head Mounted Wearable Eye Tracking System for Dynamical Calibration

In this work, a new head mounted eye tracking system is presented. Based on computer vision techniques, the system integrates eye images and head movement, in real time, performing a robust gaze point tracking. Nystagmus movements due to vestibulo-ocular reflex are monitored and integrated. The system proposed here is a strongly improved version of a previous platform called HATCAM, which was robust against changes of illumination conditions. The new version, called HAT-Move, is equipped with accurate inertial motion unit to detect the head movement enabling eye gaze even in dynamical conditions. HAT-Move performance is investigated in a group of healthy subjects in both static and dynamic conditions, i.e. when head is kept still or free to move. Evaluation was performed in terms of amplitude of the angular error between the real coordinates of the fixed points and those computed by the system in two experimental setups, specifically, in laboratory settings and in a 3D virtual reality (VR) scenario. The achieved results showed that HAT-Move is able to achieve eye gaze angular error of about 1 degree along both horizontal and vertical direction

Dominant Lyapunov exponent and approximate entropy in heart rate variability during emotional visual elicitation

In this work we characterized the non-linear complexity of Heart Rate Variability (HRV) in short time series. The complexity of HRV signal was evaluated during emotional visual elicitation by using Dominant Lyapunov Exponents (DLEs) and Approximate Entropy (ApEn). We adopted a simplified model of emotion derived from the Circumplex Model of Affects (CMAs), in which emotional mechanisms are conceptualized in two dimensions by the terms of valence and arousal. Following CMA model, a set of standardized visual stimuli in terms of arousal and valence gathered from the International Affective Picture System (IAPS) was administered to a group of 35 healthy volunteers. Experimental session consisted of eight sessions alternating neutral images with high arousal content images. Several works can be found in the literature showing a chaotic dynamics of HRV during rest or relax conditions. The outcomes of this work showed a clear switching mechanism between regular and chaotic dynamics when switching from neutral to arousal elicitation. Accordingly, the mean ApEn decreased with statistical significance during arousal elicitation and the DLE became negative. Results showed a clear distinction between the neutral and the arousal elicitation and could be profitably exploited to improve the accuracy of emotion recognition systems based on HRV time series analysis

Functional assessment of bidirectional cortical and peripheral neural control on heartbeat dynamics: A brain-heart study on thermal stress

The study of functional Brain-Heart Interplay (BHI) from non-invasive recordings has gained much interest in recent years. Previous endeavors aimed at understanding how the two dynamical systems exchange information, providing novel holistic biomarkers and important insights on essential cognitive aspects and neural system functioning. However, the interplay between cardiac sympathovagal and cortical oscillations still has much room for further investigation. In this study, we introduce a new computational framework for a functional BHI assessment, namely the Sympatho-Vagal Synthetic Data Generation Model, combining cortical (electroencephalography, EEG) and peripheral (cardiac sympathovagal) neural dynamics. The causal, bidirectional neural control on heartbeat dynamics was quantified on data gathered from 26 human volunteers undergoing a cold-pressor test. Results show that thermal stress induces heart-to-brain functional interplay sustained by EEG oscillations in the delta and gamma bands, primarily originating from sympathetic activity, whereas brain-to-heart interplay originates over central brain regions through sympathovagal control. The proposed methodology provides a viable computational tool for the functional assessment of the causal interplay between cortical and cardiac neural control

Toward brain-heart computer interfaces: A study on the classification of upper limb movements using multisystem directional estimates

Objective. Brain-computer interfaces (BCIs) exploit computational features from brain signals to perform a given task. Despite recent neurophysiology and clinical findings indicating the crucial role of functional interplay between brain and cardiovascular dynamics in locomotion, heartbeat information remains to be included in common BCI systems. In this study, we exploit the multidimensional features of directional and functional interplay between electroencephalographic and heartbeat spectra to classify upper limb movements into three classes. Approach. We gathered data from 26 healthy volunteers that performed 90 movements; the data were processed using a recently proposed framework for brain-heart interplay (BHI) assessment based on synthetic physiological data generation. Extracted BHI features were employed to classify, through sequential forward selection scheme and k-nearest neighbors algorithm, among resting state and three classes of movements according to the kind of interaction with objects. Main results. The results demonstrated that the proposed brain-heart computer interface (BHCI) system could distinguish between rest and movement classes automatically with an average 90% of accuracy. Significance. Further, this study provides neurophysiology insights indicating the crucial role of functional interplay originating at the cortical level onto the heart in the upper limb neural control. The inclusion of functional BHI insights might substantially improve the neuroscientific knowledge about motor control, and this may lead to advanced BHCI systems performances

cvxEDA: a Convex Optimization Approach to Electrodermal Activity Processing

This paper reports on a novel algorithm for the analysis of electrodermal activity (EDA) using methods of convex optimization. EDA can be considered one of the most common observation channels of sympathetic nervous system activity, and manifests itself as a change in electrical properties of the skin, such as skin conductance (SC). The proposed model describes SC as the sum of three terms: the phasic component, the tonic component, and an additive white Gaussian noise term incorporating model prediction errors as well as measurement errors and artifacts. This model is physiologically inspired and fully explains EDA through a rigorous methodology based on Bayesian statistics, mathematical convex optimization and sparsity. The algorithm was evaluated in three different experimental sessions to test its robustness to noise, its ability to separate and identify stimulus inputs, and its capability of properly describing the activity of the autonomic nervous system in response to strong affective stimulation. Results are very encouraging, showing good performance of the proposed method and suggesting promising future applicability, e.g. in the field of affective computing

Complexity index from a personalized wearable monitoring system for assessing remission in mental health

This study discusses a personalized wearable monitoring system, which provides information and communication technologies to patients with mental disorders and physicians managing such diseases. The system, hereinafter called the PSYCHE system, is mainly comprised of a comfortable t-shirt with embedded sensors, such as textile electrodes, to monitor electrocardiogram-heart rate variability (HRV) series, piezoresistive sensors for respiration activity, and triaxial accelerometers for activity recognition. Moreover, on the patient-side, the PSYCHE system uses a smartphone-based interactive platform for electronic mood agenda and clinical scale administration, whereas on the physician-side provides data visualization and support to clinical decision. The smartphone collects the physiological and behavioral data and sends the information out to a centralized server for further processing. In this study, we present experimental results gathered from ten bipolar patients, wearing the PSYCHE system, with severe symptoms who exhibited mood states among depression (DP), hypomania(HM), mixed state (MX), and euthymia (EU), i.e., the good affective balance. In analyzing more than 400 h of cardiovascular dynamics, we found that patients experiencing mood transitions from a pathological mood state (HM, DP, or MX - where depressive and hypomanic symptoms are simultaneously present) to EU can be characterized through a commonly used measure of entropy. In particular, the SampEn estimated on long-term HRV series increases according to the patients' clinical improvement. These results are in agreement with the current literature reporting on the complexity dynamics of physiological systems and provides a promising and viable support to clinical decision in order to improve the diagnosis and management of psychiatric disorders